Skip to content

Conversation

@dlevy-msft-sql
Copy link
Contributor

@dlevy-msft-sql dlevy-msft-sql commented Dec 1, 2025

Work Item / Issue Reference

GitHub Issue: #352


Summary

This PR fixes cursor.description to properly return SQL type codes per DB-API 2.0 specification, while maintaining backwards compatibility with libraries like pandas and polars that compare against Python types.

The original issue was reported when using polars pl.read_database() which failed with:

ComputeError: could not append value: 2013-01-01 of type: date to the builder

Key Changes

1. New SqlTypeCode Class

A dual-compatible type code that compares equal to both:

  • SQL type integers (e.g., desc[1] == -9) for DB-API 2.0 compliance
  • Python types (e.g., desc[1] == str) for pandas/polars/library compatibility
cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas/polars compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

2. SQL Server Spatial Type Support

Added handling for SQL Server User-Defined Types (SQL_SS_UDT = -151):

  • geography (e.g., GPS coordinates, regions)
  • geometry (e.g., shapes, planar coordinates)
  • hierarchyid (e.g., org charts, tree structures)

3. ODBC 3.x Date/Time Type Codes

Added support for:

  • SQL_TYPE_DATE (91)
  • SQL_TYPE_TIME (92)
  • SQL_TYPE_TIMESTAMP (93)
  • SQL_TYPE_TIMESTAMP_WITH_TIMEZONE (95)

4. Hash/Equality Contract Fix

Made SqlTypeCode intentionally unhashable since __eq__ compares to multiple types with different hash values. Attempting to hash raises TypeError with a helpful message guiding users to use int(type_code) as dict keys instead.


Files Changed

File Changes
mssql_python/cursor.py Added SqlTypeCode class, updated _initialize_description(), fixed _build_converter_map()
mssql_python/constants.py Added SQL_SS_UDT, SQL_SS_TIME2, SQL_SS_XML constants
mssql_python/pybind/ddbc_bindings.cpp Added C++ handling for spatial types, datetime2, smalldatetime
mssql_python/__init__.py Exported SqlTypeCode
mssql_python/mssql_python.pyi Added type stubs for SqlTypeCode
mssql_python/connection.py Updated converter methods to accept SqlTypeCode
tests/test_002_types.py Added 18 SqlTypeCode unit tests
tests/test_004_cursor.py Added cursor description tests, thread safety tests
tests/test_017_spatial_types.py NEW: 37 spatial type tests (geography, geometry, hierarchyid)
tests/test_018_polars_integration.py NEW: 6 polars integration tests
requirements.txt Added polars for integration testing
CHANGELOG.md Documented changes

Usage Examples

SqlTypeCode - Dual Compatibility

cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas/polars compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

Polars Integration (Original Issue)

import polars as pl
import mssql_python

conn = mssql_python.connect("Server=myserver;Database=mydb;...")

# This now works correctly!
df = pl.read_database(
    query="SELECT * FROM Sales.Customers",
    connection=conn
)
print(df)

Spatial Types

# Geography (GPS coordinates)
cursor.execute("""
    INSERT INTO locations (geo) 
    VALUES (geography::STGeomFromText('POINT(-122.349 47.651)', 4326))
""")

# Fetch as binary
cursor.execute("SELECT geo FROM locations")
geo_bytes = cursor.fetchone()[0]  # Returns bytes

# Or fetch as WKT text
cursor.execute("SELECT geo.STAsText() FROM locations")
wkt = cursor.fetchone()[0]  # Returns "POINT (-122.349 47.651)"

Testing

Test Coverage

  • ✅ All existing tests pass (1425 passed)
  • ✅ 18 new SqlTypeCode unit tests
  • ✅ 37 new spatial type tests (geography, geometry, hierarchyid)
  • ✅ 6 new polars integration tests
  • ✅ Thread safety tests for _column_metadata
  • ✅ Binary parameter round-trip tests for spatial types
  • 📊 Diff Coverage: 93%
  • 📊 Overall Coverage: 77%

Polars Integration Tests

Test Description
test_polars_read_database_basic Basic data types (INT, NVARCHAR, FLOAT)
test_polars_read_database_with_dates DATE columns - the original failure case
test_polars_read_database_all_common_types All common SQL Server types
test_polars_read_database_with_nulls NULL value handling
test_polars_read_database_large_result 1000 row result set
test_cursor_description_type_code_polars_compatibility Direct type comparison verification

Breaking Changes

None. The SqlTypeCode class maintains full backwards compatibility:

  • cursor.description[i][1] == str still works
  • cursor.description[i][1] == -9 also works
  • int(cursor.description[i][1]) returns raw SQL type code

Related Issues

Copilot AI review requested due to automatic review settings December 1, 2025 17:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes issue #352 by implementing two main improvements: (1) correcting cursor.description to return SQL type codes (integers) instead of Python type objects, ensuring DB-API 2.0 specification compliance, and (2) adding support for SQL Server spatial data types (geography, geometry, hierarchyid) by handling the SQL_SS_UDT type code (-151).

Key changes:

  • Fixed cursor.description[i][1] to return SQL type integer codes (e.g., 4, -9) instead of Python types (int, str) per DB-API 2.0 spec
  • Added SQL_SS_UDT (-151) support for SQL Server User-Defined Types including spatial data types
  • Updated output converter lookup to use SQL type codes consistently throughout the codebase

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
mssql_python/cursor.py Changed cursor.description to return SQL type codes instead of Python types; added _column_metadata storage; updated _build_converter_map to use SQL type codes; added mappings for UDT, XML, DATETIME2, SMALLDATETIME types
mssql_python/constants.py Added SQL_SS_UDT = -151 constant for SQL Server User-Defined Types
mssql_python/pybind/ddbc_bindings.cpp Added C++ constants for SQL_SS_UDT, SQL_DATETIME2, SQL_SMALLDATETIME; implemented UDT handling in SQLGetData_wrap, FetchBatchData, FetchMany_wrap, and FetchAll_wrap for LOB streaming
tests/test_004_cursor.py Added lob_wvarchar_column to test schema; updated test_cursor_description to expect SQL type codes; added comprehensive geography type test suite (14 new tests); separated LOB and non-LOB fetch tests; fixed output converter test for UTF-16LE encoding
tests/test_003_connection.py Simplified converter integration tests to use SQL type constants directly instead of dynamic type detection

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dlevy-msft-sql dlevy-msft-sql added bug Something isn't working pr-size: medium Moderate update size labels Dec 1, 2025
@github-actions
Copy link

github-actions bot commented Jan 5, 2026

📊 Code Coverage Report

🔥 Diff Coverage

93%


🎯 Overall Coverage

77%


📈 Total Lines Covered: 5556 out of 7213
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql_python/init.py (100%)
  • mssql_python/connection.py (88.2%): Missing lines 50,1020
  • mssql_python/constants.py (100%)
  • mssql_python/cursor.py (90.6%): Missing lines 134,158,1132,1479,2547
  • mssql_python/pybind/ddbc_bindings.cpp (100%)

Summary

  • Total: 100 lines
  • Missing: 7 lines
  • Coverage: 93%

mssql_python/connection.py

Lines 46-54


Lines 1016-1024


mssql_python/cursor.py

Lines 130-138


Lines 154-162


Lines 1128-1136


Lines 1475-1483


Lines 2543-2551



📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.5%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.row.py: 79.5%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.__init__.py: 84.9%
mssql_python.cursor.py: 85.2%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@dlevy-msft-sql dlevy-msft-sql added pr-size: small Minimal code update pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Jan 16, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

tests/test_004_cursor.py:1

  • Changed from catching generic Exception to specific mssql_python.Error. This is more precise and follows best practices for error handling in tests.
"""

tests/test_004_cursor.py:1

  • The date '2025-11-26' is in the future. Consider using a past date or current date to avoid potential timezone-related test failures.
"""

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Improve __hash__ error message with helpful guidance for dict key usage
- Move spatial tests (geography/geometry/hierarchyid) to test_017_spatial_types.py
- Add binary parameter round-trip tests for geography and geometry
- Update type stub and unit test for new hash behavior
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

tests/test_004_cursor.py:1

  • This change corrects the exception type from a generic Exception to mssql_python.Error, which is more specific and accurate for database errors. This is already fixed in the updated code.
"""

tests/test_004_cursor.py:13063

  • Simplified cleanup by removing unnecessary try-except block around DROP TABLE statement. This is already fixed in the updated code.
        cursor.execute(f"DROP TABLE IF EXISTS {table_name}")

tests/test_004_cursor.py:1

  • Added proper encoding handling for string comparison in output converter test. The driver passes string values as UTF-16LE encoded bytes to output converters, so the test now correctly encodes the comparison value. This is already fixed in the updated code.
"""

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@saurabh500
Copy link
Contributor

SqlTypeCode should be used instead of SQLTypeCode. I am assuming this is a public type and the naming would matter more. PascalCase for Python classes is recommended and applicable to acronyms also.

- Rename class from SQLTypeCode to SqlTypeCode (PascalCase for acronyms)
- Fix spatial binary round-trip tests to use Deserialize() instead of STGeomFromWKB()
- Update all references in cursor.py, connection.py, __init__.py, type stubs, tests, and docs
@dlevy-msft-sql
Copy link
Contributor Author

SqlTypeCode should be used instead of SQLTypeCode. I am assuming this is a public type and the naming would matter more. PascalCase for Python classes is recommended and applicable to acronyms also.

Fixed.

@dlevy-msft-sql dlevy-msft-sql marked this pull request as draft February 3, 2026 01:42
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Add test_018_polars_integration.py with 6 tests verifying polars compatibility
- Tests cover: basic types, DATE columns (original failure case), all common types, NULLs, large results
- Add polars to requirements.txt for CI
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Use pytest.importorskip() to gracefully skip tests when polars
cannot be imported due to platform incompatibility (e.g., missing
CPU instructions on Alpine ARM64).
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Removed 'decimal' and 'mssql_python' imports that were not used in the
test module. Addresses Copilot review comment.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dlevy-msft-sql dlevy-msft-sql marked this pull request as ready for review February 3, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working pr-size: medium Moderate update size

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cursor.description returning incorrect values

3 participants